Marker patterns can be added to objects or scenes to allow automatic systems to find correspondence between points in the world and points in camera images, and to find correspondences between points in one camera image and points in another camera image. The former has application in positioning, robotics, and augmented reality applications, the latter has application in automatic computer modeling to provide the coordinates of world points for applications of the former. Furthermore, marker patterns can be used to contain information relating to various products. For example, marker patterns printed out and mounted on a piece of equipment would allow an augmented reality system to aid a person constructing or servicing this equipment by overlaying virtual graphics with instructions over their view, with the aid of an image sensor (light capturing device such as camera, video camera, digital camera, etc) and the computer vision techniques that locate these patterns. Furthermore with camera cell phones and PDAs becoming commonly available, a marker could be used to link a user to a URL address providing access to a series of images, adverstisement ect. An other example of use includes, a robot could which navigates by detecting markers placed in its environment. Using computer vision, cameras and cameras cell phones to determine relative pose is an inexpensive and accurate approach useful to many domains.
There are several systems that use a light capturing device (film camera, video camera, digital camera, camera cell phones etc) and markers to provide these correspondences; they can be classified into active and passive systems. Active marker systems are defined by if the marker emits radiation (radio, visible light, infrared light, etc) to aid in its detection, while passive markers do not require power and only reflect or in some way modify incoming electromagnetic radiation. An analogy using human vision is traffic signals, traffic lights are active markers, and stop signs are passive markers. Passive markers are beneficial for many applications as they are typically much less complex and expensive, and require little or no maintenance. Markers can be made with a standard printer or can be composed of special material to improve detection such as retro-reflective material or material with shading differences only visible in infrared. Often, a passive marker is just a special pattern printed out on paper. However, while passive systems can be simpler and less expensive in hardware requirements, they need more sophisticated systems such as computer vision to detect and identify the markers.
Examples of passive markers include systems using a rigid constellation of small spheres. Others use flat patterns visible to the human eye with bi-tonal (black and white), varying shades of grey, and color patterns. These are not always very effective because of their reliance on the lighting conditions and specific light gathering device used, for example systems relying on recognizing color in markers typically need more expensive video cameras, need to control the lighting so the spectral content and brightness does not vary, and often need to not change the cameras or their positions. Likewise other systems that rely on the shades of grey in the marker could require the entire setup of lighting, markers, and cameras need to be carefully controlled similar to a studio setting, require much adjustment to function, and typically cease to function when the conditions change.
A useful fiducial marker system should function under any reasonable conditions of varying lighting, using almost any available image sensor to make it a useful product.
Existing fiducial marker systems such as ARToolkit, ARToolkit Plus, Matrix, Binary Square Marker, cybercode suffer from this susceptibility to lighting. This susceptibility is due to the binary thresholding algorithm used by these and other marker systems, the weakness lies in their common approach of attempting to label all image pixels as belonging to one or another set, these systems attempt to classify parts of the image by some method to attempt to recognize different levels of reflectance from the marker and its surroundings.
Typically these marker systems attempt to divide the image into light and dark regions, they have an intermediate stage called a binary image of same dimensions as the input image where every pixel is either labeled as white or black. Following this morphology operations are applied to group the pixels into connected regions, each white or black pixel is then labeled by an object number it belongs to.
Reactivision is a system that uses the topology information of connected regions of light and dark inside one another to recognize a marker, most other systems use the contours of these connected regions to find some unique feature such as a polygon border. This step of attempting to separate different levels of reflectance of objects and markers in the image sensor by labeling the received intensity levels at image pixels is a weak point, often they attempt to simply classify white pixels from black by applying a threshold to the image or a region. This often fails because it is often not possible to separate objects or marker sections by applying a brightness or color threshold. These systems do not function well because of the assumption that parts of markers with equal reflectance properties (shading or coloring) will manifest themselves as regions of uniform incoming light to an image sensor.
Most passive marker systems rely on a two step process; firstly locating a unique feature such as a geometric feature, and secondly by verification and identification step to determine if it is indeed a marker and which one it is out of a library of possible markers. Often marker systems fail in the first step due to their reliance on the binarization method described above.
Another source of weakness for existing marker systems is their lack of robust processing in the verification and identification stages. Most other marker systems use digital patterns encoded in the marker and do the verification and identification by processing an extracted set of digital symbols. Often the digital symbols found in the image are simply compared to a set of expected symbol sets to verify if it is a marker. A superior method is to use a checksum, to allocate some of the digital symbols that can be decoded to verify data integrity, such as the CRC-16 16-bit checksum or a simply parity check with one checksum symbol. Some use checksums such as Matrix and Binary Square Marker which gives them more robust performance, but these systems still fall short of practical usability because of their use of the binarization method, and because of an insufficient amount of information in the marker. Binary Square marker only uses a grid of 12 information carrying cells in the marker which cannot convey much information to allow reliable detection.
The act of extracting a set of digital symbols from the image sensor's array of pixels can sometimes introduce errors due to sources such as image sensor noise, lighting variances, or partial occlusion (blocking) of objects. Error detection and correction algorithms, can recognize and repair some errors at the cost of requiring more digital symbols. Current marker systems do not use both error correction and checksums which greatly reduces their effectiveness.
Also of consideration for a fiducial marker system is the ability to function with the geometric distortion of a perspective projection image sensor, which represents the most common type of film, digital, and video cameras available. The application of the mathematical method of homographies allows this distortion to be accounted for by using the position of at least four non-collinear points in the image if both the marker pattern and the image sensor are flat planes and the image capture can be modeled with the pinhole model. 2D barcode systems such as Datamatrix, QR, Maxicode and Cybercode (FIG. 1) are not suitable as fiducial markers because they provide less than four such salient points. They are intended for situations with a narrow field of view or placement parallel to the image plane such that their projection in the image can be approximated by an affine projection only requiring three such points. Datamatrix illustrated in FIG. 1a has the L-shaped located for which the two ends and the corner are 3 points used to define an “affine” transform to decode the digital pattern, likewise QR illustrated in FIG. 1b has 3 square blocks in the corner for the same purpose.
Finally, existing systems that use digital patterns do not provide proper level of information encoded in the marker. They either provide too little as with Binary Square Marker, Matrix, SCR, HOM, IGD, Canon, or provide too much such as the 2D barcode systems like Cybercode. There has to be enough information to encode enough uniqueness to have a sufficiently large library of potential markers and to have symbols allocated for a checksum and for error correction. Also the information encoded has to be small enough that the marker can be detected with a minimum small number of image pixels. The latter requirement affects the range of distance a given marker can be detected at. One purpose of a fiducial marker system is to find correspondences, another purpose would be to carry information. They are designed to be detectable with their size in the image being as small as possible to give the greatest range the marker system can be used at. Thus, the marker should be as small as possible to give the greatest detection range but be large enough to contain enough information to allow it to be reliably detected.
The issues discussed above are now defined more precisely as parameters necessary for a reliable and useful markers systems. Some numerical metrics for appraising marker systems are;                1) the false positive rate,        2) the inter-marker confusion rate,        3) the false negative rate,        4) the minimal marker size,        5) the vertex jitter characteristics,        6) the marker library size, and        7) the speed performance.        
The false positive rate is the rate of falsely reporting the presence of a marker when none is present. The inter-marker confusion rate is the rate of when a marker is detected, but the wrong id was given, i.e. one marker was mistaken for another. The false negative rate is the probability that a marker is present in an image but not reported.
Another metric is 4) the minimal marker size which is the size in pixels required for reliable detection as discussed above. Speed performance is a consideration, a vision based fiducial marker tracking system must be real time with available computing power and thus the detecting computer vision algorithm must be efficient enough to use existing technology. Two other factors of fiducial marker system performance, not quantitatively measured herein but described loosely, are the immunity to lighting conditions and occlusion. Occlusion is when a fiducial marker is partially or completely blocked from view by the camera by some other object, a marker system should be able to still detect a marker despite partial occlusion. A robust marker system for industry should be able to detect markers with widely varying lighting.
The existing systems only typically exist in laboratories due to their weakness in some or all of the above categories, their lack of robustness discounts them from wide implementation in industry and society.